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ABSTRACT 



The invention provides an identification data insertion and 
detection system for digital data which improves the detec- 
tion ratio when an image is converted into frequency com- 
ponents and electronic watermark data are embedded in a 
portion of an image having a strong frequency component 
and besides allows a statistical similarly to be calculated 
with a higher degree of accuracy. The identification data 
insertion system for digital data includes a DCT calculator 
for extracting blocks of 8x8 pixels from within a current 
image and performing DCT (discrete cosine transform) 
calculation for the blocks, a quantizer for quantizing data 
outputted from the DCT calculator, a quantization table for 
being referred to when the quantizer performs quantization, 
an electronic watermark data insertion element for inserting 
electronic watermark data into data after quantization, and 
an encoder for encoding data outputted from the electronic 
watermark data insertion element to produce MPEG data in 
which the electronic watermark data are inserted and out- 
putting the MPEG data. 

4 Claims, 8 Drawing Sheets 
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IDENTIFICATION DATA INSERTION AND 
DETECTION SYSTEM FOR DIGITAL DATA 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

This invention relates to a system for processing a digital 
image, and more particularly to a system for embedding 
electronic watermark data as identification data having spe- 
cial information into a digital image. 

2. Description of the Related Art 
In recent years, illegal duplication of a digital image has 

become a serious problem. 

In order to prevent illegal duplication, a system has been 
proposed wherein digital image data are ciphered such that 
only a reproduction system having a legal decipherment key 
can reproduce the ciphered digital image data. However, 
once the cipher is deciphered, later duplication cannot be 
prevented. 

Thus, in order to prevent illegal use and duplication of a 
digital image, a method has been proposed wherein elec- 
tronic watermark data which are special information are 
embedded into the digital image itself. 

As such electronic watermark data for a digital image as 
described above, two kinds of electronic watermark data of 25 
visible electronic watermark data and invisible electronic 
watermark data are available. 

The visible electronic watermark data are used such that 
a special character or symbol or the like is combined with an 
image so that it may be visually recognized. While the 30 
visible electronic watermark data deteriorate the picture 
quality, they have an effect of visually appealins to a user of 
a digital image for prevention of illegal use of the digital 
image. 

An example of embedding of visible electronic watermark 
data is disclosed in Japanese Patent Laid-Open Application 
No. Heisei 8-241403. According to the method disclosed in 
the document just mentioned, visible electronic watermark 
data are combined with an original image such that only the 
brightness components of those pixels which correspond to 
an opaque portion of the electronic watermark data are 
varied while color components of the pixels are not varied. 
In this instance, the scaling value by which the brightness 
components of pixels are varied is determined based on 
color components, random numbers, values of pixels of the 
electronic watermark data and so forth. 

On the other hand, the invisible electronic watermark data 
are embedded into an image so that the picture quality may 
not be deteriorated. Since the invisible electronic watermark 
data little deteriorate the picture quality, they are character- 
ized in that they cannot be recognized visually. 

However, if special information which allows identifica- 
tion of an author is embedded as the electronic watermark 
data, then also after illegal duplication is performed, the 
author can be specified by detecting the electronic water- 
mark data. On the other hand, if duplication inhibition 
information is embedded, then, for example, when a repro- 
duction apparatus detects the duplication inhibition 
information, it is possible to notify the user that duplication 
of the data is inhibited or to render a duplication prevention 
mechanism in the reproduction apparatus operative to limit 
duplication of the data by a video tape recorder ( VTR) or the 
like. 

One available methods of embedding invisible electronic 
watermark data into a digital image is to embed special 
information as electronic watermark data into a portion of 
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pixel data such as a least significant bit (LSB) which has a 
comparatively little influence on the picture quality. 
However, where this method is applied, electronic water- 
mark data can be removed readily from an image. For 
example, if a low-pass filter is used, then information of the 
LSB of a pixel is lost. Further, electronic watermark data are 
lost also by image compression processing since the image 
compression processing reduces the data amount by remov- 
ing information at such a portion which has a comparatively 
little influence on the picture quality as mentioned above. 
Consequently, it is difficult to re -detect the electronic water- 
mark data. 

Another method is disclosed in Japanese Patent Laid- 
Open Application No. Heisei 6-315131. 

In the method of the document just mentioned, a region 
which can be replaced, upon reproduction, by a peripheral 
region without causing deterioration of an image is detected 
using a correlation of images of successive frames, and the 
level of the region of an object of conversion is converted to 
embed particular information. In this method, upon 
reproduction, the region in which identification data are 
embedded is specified using the signal missing portion and 
the replacement information, and the portion is corrected to 
restore the image. 

A further method is disclosed in Japanese Patent Laid- 
Open Application No. Heisei 5-30466 wherein a video 
signal is frequency converted and information having a 
frequency signal lower than a frequency band of the video 
signal after the frequency conversion is embedded into the 
video signal. In this method, the original video signal is 
extracted using a high pass filter while embedded identifi- 
cation data are extracted using a low pass filter. 

As another method which involves frequency conversion 
of an image, a method has been proposed wherein an image 
is frequency converted and electronic watermark data are 
embedded into a region of the video signal after the fre- 
quency conversion in which strong frequency components 
are included (Nikkei Electronics. Apr. 22, 1996, No. 660, 
p.13). 

In this method, since electronic watermark data are 
embedded into frequency components, the electronic water- 
mark data are not lost even by image processing such as 
compression processing or filtering. Further, random num- 
bers which exhibit a normal distribution are adopted as the 
electronic watermark data to prevent interference between 
electronic watermark data and make it difficult to destroy 
electronic watermark data without having a significant influ- 
ence on the entire image. 

According to the electronic watermark data embedding 
method in the method described above, an original image 
501 is converted into frequency components using a DCT 
(discrete cosine transform) transformer 502 as seen in FIG. 
5. Then, n data which exhibit comparatively high values in 
a frequency region are selected and individually represented 
by f(l), f(2), . . . , f(n), and electronic watermark data 503 
(w(l), w(2), . . . , w(n)) are selected from within a normal 
distribution whose average is 0 and whose dispersion is 1. 
Then, F(i)-f(i)+ax|f(i)|xw(i) is calculated for each i by an 
electronic watermark data insertion element 504. Here, a is 
a scaling factor. Finally, DCT transform coefficients includ- 
ing those frequency components in which f(i) are replaced 
by F(i) are inverse transformed by an inverse DCT trans- 
former 509 to obtain an image 510 in which the electronic 
watermark data is embedded. 

Electronic watermark data is detected using the following 
method. In the detection method, an original image and 
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electronic watermark data candidates w(i) (where i«l, w(i) and WF(i) is calculated in accordance with OWFxw/ 

2, . . . , n) must be known. (WFDxwD) making use of an inner product of vectors. 

Referring to FIG. 6, an original image 601 and an dec- Here, W«(WF(1), WF(2), . . . , WF(n)), w«(w(l), w(2), 

tronic watermark data containing image 602 are converted w(n)), WFD-absolute value of the vector WF, and 

into frequency components using DCT transformers 603 and 5 wD=absolute value of the vector w. When the statistical 

604, respectively. In a frequency region, values of factors similarity C has a value higher than a particular value, it is 

corresponding to f(l), f(2), . . . , f(n) in which electronic discriminated that the electronic watermark data are embed- 

watermark data are embedded are represented by F(l), ded. 

F(2), . . . , F(n), respectively. An electronic watermark data FIG. 7 shows a conventional system which embeds elec- 

extractor 605 extracts electronic watermark data W(i) based io tronic watermark data using such a method as described 

on f(i) and F(i) by calculation of W(i)«(F(i)-f(i))/f(i). Then, above, and FIG, 8 shows a conventional system which 

an inner product calculator 608 calculates, based on elec- detects electronic watermark data embedded by such a 

tronic watermark data 609, a statistical similarity between system as shown in FIG, 7. 

w(i) and W(i) in accordance with C-Wxw/(WDxwD) using Referring first to FIG. 7, in the system shown, a current 
inner products of vectors. Here, W=(W(1), W(2), . . . , 15 i ma gc 701 is divided into blocks 702 of 8x8 pixels and 
W(n)), w-(w(l), w(2) , . . . , w(n)), WD -absolute value of the discrete cosine transformed by a DCT transformer 703, and 
vector W, and wD=absolute value of the vector w. A statis- men electronic watermark data 704 arc inserted into an 
tical similarity discriminator 610 discriminates, when the output of the DCT transformer 703 by an electronic water- 
statistical similarity C is higher than a particular value, that maT ^ ^ata insertion element 705. In the electronic watermark 
the pertaining electronic watermark data are embedded. 20 j ala mser ti on element 705, partial averages mentioned 

If electronic watermark data are embedded in an image above are calculated by a partial average calculator 706, and 

using the method described above, then they can be effee- a result of the calculation and the electronic watermark data 

tively used when an author who owns the original image 704 are multiplied by a multiplier 708. Then, an output of 

performs detection processing for digital image data which the multiplier 708 is multiplied by a constant (scaling factor) 

arc supposed to be an illegal duplicate. 25 a 707 by another multiplier 709. Then, a result of the 

However, since the method described above requires an multiplication is added to the image data from the DCT 

original image, although it is effective when an author who transformer 703 by an adder 710 to make image data in 

owns an original image performs detection processing for which the electronic watermark data are embedded. The 

image data which are supposed to be an illegal duplicate, a image data are quantized by a quantizer 711 with reference 

reproduction apparatus of a terminal cannot perform elec- 30 to a quantization table 712 and then encoded by an encoder 

tronic watermark data detection processing because it does 713 to produce MPEG image data 714 in which the elec- 

not have the original image. Thus, another method which is tronic watermark data are embedded, and the MPEG image 

an improvement over the method described above and is data 714 are outputted, 

directed to terminal processing and particularly to an MPEG ^ Referring now to FIG. 8, in the system shown, MPEG 

(Moving Picture Experts Group) system has been proposed. image data 801 in which electronic watermark data are 

According to the improved method, an original image is embedded are decoded by a decoder 802 and then dequan- 

divided into blocks of 8 pixelsx8 pixels, and embedding and tized by a dequantizer 803, whereafter they are inverse DCT 

extraction processing of electronic watermark data is per- transformed by an inverse DCT calculator 804 to restore 

formed regarding each block as a processing unit. 4Q original image data 805. Meanwhile, electronic watermark 

In electronic watermark data embedding processing, data data are extracted from the output of the dequantizer 803 by 

are first set successively as f(l), f(2), . . . , f(n) in the an electronic watermark data extractor 806. In the electronic 

ascending order of the frequency component of an AC watermark data extractor 806, after partial average calcula- 

component in a frequency region after discrete cosine trans- lion by a partial average calculator 807 and division by a 

form in MPEG coding processing is completed, and elec- 45 divider 808 are performed, sum totals WF(i) for one screen 

tronic watermark data w(l),w(2),...,2(n) are selected from are performed for each i by an adder 809. Then, inner 

within a normal distribution whose average is 0 and whose products of the sum totals W(i) and electronic watermark 

dispersion is 1, and then F(i)=f(i)+axavg(f(i))xw(i) is per- data 811 of an object of detection are calculated by an inner 

formed for each i. Here, a is a scaling factor, and avg(f(i)) product calculator 810 to obtain a statistical similarity 812. 

is a partial average which is an average of absolute values of 50 Then, if the statistical similarity 812 has a value higher than 

three neighboring points to f(i). Then, succeeding processing a particular value, then it is discriminated that the electronic 

of MPEG coding processing is performed replacing f(i) with watermark data are embedded in the MPEG image data 801. 

F(i). In the method disclosed in Japanese Patent Laid-Open 

Detection of electronic watermark data is performed in Application No. Heisei 6-315131, since electronic water- 
accordance with the following method. In this detection 55 mark information is not embedded in all frames, illegal 
method, an original image is not required, but only it is duplication cannot be prevented with regard to any frame in 
required that data candidates w(i) (where i-1, 2, . . . , n) be which the electronic watermark information is not embed- 
known ded. Further, since successive frames represent still pictures 

In a frequency region of each block after dequantization and it is presupposed that there is no variation between 

of MPEG decompression processing is completed, data are 60 successive frames, where moving pictures which nvolve 

represented as F(l), F(2) F(n) in the ascending order strong movements are handled, a region into which elec- 

of the frequency component. An average value of absolute ironic watermark data are to be embedded cannot be speci- 

values of three neighboring points to F(i) is represented as ned. Consequently, electronic watermark cannot be embed- 

a partial average avg(F(i)), and electronic watermark data ded. 

W(i) is calculated in accordance with W(i)«F(i)/avg(F(i)). 65 Meanwhile, in the method disclosed in Japanese Patent 

Further, a sum total WF(i) of W(i) for one image is calcu- Laid-Opcn No. Heisci 5-30466, since electronic watermark 
lated for each i. Thereafter, a statistical similarity between data are embedded into a portion of an image which is lower 
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than a frequency region after frequency conversion of the factor of the data outputted from the quantizer as a partial 

image, the electronic watermark data can be removed readily average, a first multiplier for multiplying the electronic 

using a band-pass filter. watermark data by an output of the partial average calculator 

Further, with the method wherein electronic watermark for the individual factors, a second multiplier for multiply- 

data arc embedded into a portion having a strong frequency 5 *g each factor of an output of the first multiplier by a 

. c. c • u-1 i * • constant to be used to vary the size of the electronic 

component after frequency conversion, while electronic . , . . , . t j , f 

A r . , 4 * u j * en lL ,-, watermark data to be inserted, and an adder for adding an 

watermark data cannot be removed using ; afilter or the like, ^ secQnd mm ^ aQd ^ QU of the ^ 

electronic watermark data inserted after DCTprocessing are ^ fof me individual fac [ ors 

sometimes lost by quantization process According to another aspect of the present invention, 

where a value of any frequency component after DCI falls io ^ . & ^ aQ idenUfi( f ation data 6 £ cction system for 

within a fixed range, it is converted into a particular repre- data ad ^ tQ detect dectronic watermark data 

sentative value in quantization processing. in ^ rted by ^ ideDtification data insertion syslem for digita , 

Accordingly, due to an action of quantization, the values data deS cribed above and comprising a decoder for decoding 

of a partial average avg(f(i)) of three neighboring points compressed image data, a dequantizer for dequantizing data 

upon insertion and a partial average avg(F(i)) of the three 15 0Ut p Utte d from the decoder, an inverse DCT calculator for 

neighboring points upon detection are sometimes different performing inverse discrete cosine transform calculation for 

by a large amount. In such an instance, the statistical data outputted from the dequantizer and outputting image 

similarity between extracted electronic watermark data and dala obtained by the inverse discrete cosine transform 

electronic watermark data to be detected exhibits a low calculation, an electronic watermark data extractor for 

value and makes a factor which causes an erroneous dis- 20 extracting and outputting, from among the data outputted 

crimination result. f rom me decoder, those data which are supposed to be 

SUMMARY OF THE INVENTION electronic watermark data, an adder for adding data in units 

of a block of 8x8 pixels outputted from the electronic 

It is an object of the present invention to provide an watermark data extractor for individual factors over one 

identification data insertion and detection system for digital screen, and an inner product calculator for calculating inner 

data which improves the detection ratio when an image is products of electronic watermark data to be detected and the 

converted into frequency components and electronic water- data outputted from the adder to calculate a statistical 

mark data are embedded in a portion of an image having a similarity and outputting the statistical similarity, 

strong frequency component. ^ Thc electronic watermark data extractor may include a 

It is another object of the present invention to provide an partial average calculator for calculating an average of 

identification data insertion and detection system for digital absolute values of three neighboring points of each of data 

data which allows a statistical similarly to be calculated with m um ts of a matrix block of 8x8 pixels outputted from thc 

a higher degree of accuracy. decoder as a partial average, and a divider for dividing the 

In order to attain the objects described above, according 35 data outputted from the decoder by the partial average 
to the present invention, taking it into consideration that outputted from the partial average calculator. 
DCT coefficients are rounded by quantization, insertion of With the identification data insertion system and the 
electronic watermark data is performed not in a stage identification data detection system for digital data, since 
between DCT processing and quantization processing, but electronic watermark data insertion processing is performed 
after quantization. On the other hand, upon detection, elec- 40 after quantization whereas electronic watermark data detec- 
tronic watermark data detection processing is performed tion processing is performed prior to dequantization, other- 
prior to dequantization. wise possible loss or disappearance of electronic watermark 

In the meantime, where insertion of electronic watermark data by quantization can be prevented, and the difference 

data is performed in a stage between DCT processing and between a partial average used upon insertion and another 

quantization processing similarly as in thc conventional 45 partial average used upon detection is reduced, 

methods, when a partial average is to be calculated, an Consequently, a statistical similarity can be calculated with 

absolute value of a value obtained by multiplying an integral a higher degree of accuracy. Accordingly, the accuracy in 

value obtained by division of a corresponding value of a detection of electronic watermark data is improved, 

quantization further by the corresponding value of the It is to be noted that the identification data insertion 

quantization table. 50 system and the identification data detection system for 

More particularly, according to an aspect of the present digital data may be combined so as to form a single 

invention, there is provided, an identification data insertion identification data insertion and detection system for digital 

system for digital data for inserting electronic watermark data. 

data as identification data having special information into a According to a further aspect of the present invention, 

digital image, comprising a DCT calculator for extracting ss there is provided an identification data insertion system for 

matrix blocks of 8x8 pixels from a current image and digital data, comprising a DCT calculator for extracting 

performing discrete cosine transform calculation for the matrix blocks of 8x8 pixels from a current image and 

blocks, a quantizer for quantizing data outputted from the performing discrete cosine transform calculation for the 

DCT calculator, a quantization table for being referred to blocks, an electronic watermark data insertion element for 

when the quantizer performs quantization, an electronic 60 inserting electronic watermark data into data after the dis- 

watermark data insertion element for inserting electronic crete cosine transform outputted from the DCT calculator, a 

watermark data into the data after the quantization, and an quantizer for quantizing data outputted from the electronic 

encoder for encoding data outputted from the electronic watermark data insertion element, a quantization table for 

watermark data insertion element. being referred to when the quantizer performs quantization 

The electronic watermark data insertion element may 65 and when the electronic watermark data insertion element 

include a partial average calculator for calculating an aver- calculates a partial average, and an encoder for encoding 

age of absolute values of three neighboring points of each data outputted from the quantizer. 
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The electronic watermark data insertion element may 
include a partial average calculator for calculating, using the 
quantization table when a partial average of the data out- 
putted from the DCT calculator is to be calculated, an 
average among three neighboring points of absolute values 
of values obtained each by multiplying an integer value 
obtained by dividing a value of each factor outputted from 
the DCT calculator by a corresponding value of the quan- 
tization table by the corresponding value of the quantization 
table as a partial average of the pertaining factor, a first 
multiplier for multiplying an output of the partial average 
calculator by the electronic watermark data for the indi- 
vidual factors, a second multiplier for multiplying data 
outputted from the first multiplier by a constant to be used 
to vary the size of the electronic watermark data to be 
embedded, and an adder for adding the data outputted from 
the DCT calculator and data outputted from the second 
multiplier for the individual factors. 

According to a still further aspect of the present invention, 
there is provided an identification data detection system for 
digital data adapted to detect electronic watermark data 
inserted by the identification data insertion system for digital 
data described above and comprising a decoder for decoding 
compressed image data, a dequantizer for dequantizing data 
outputted from the decoder, an inverse DCT calculator for 
performing inverse discrete cosine transform calculation for 
data outputted from the dequantizer and outputting image 
data obtained by the inverse discrete cosine transform 
calculation, an electronic watermark data extractor for 
extracting and outputting, from among the data outputted 
from the dequantizer, those data which are supposed to be 
electronic watermark data, an adder for adding data in units 
of a block of 8x8 pixels outputted from the electronic 
watermark data extractor for individual factors over one 
screen, and an inner product calculator for calculating inner 
products of electronic watermark data to be detected and the 
data outputted from the adder to calculate a statistical 
similarity and outputting the statistical similarity. 

The electronic watermark data extractor may include a 
partial average calculator for calculating an average of 
absolute values of three neighboring points of each of data 
in units of a block of 8x8 pixels outputted from the dequan- 
tizer as a partial average, and a divider for dividing the data 
outputted from the dequantizer by the partial average out- 
putted from the partial average calculator. 

With the identification data insertion system wherein 
insertion of electronic watermark data is performed in a 
stage between DCT processing and quantization processing 
similarly as in the conventional methods and with the 
identification data detection system, when a partial average 
is to be calculated, an absolute value of a value obtained by 
multiplying an integral value obtained by division of a 
corresponding value of a quantization table further by the 
corresponding value of the quantization table. Consequently, 
the difference between a partial average used upon insertion 
and another partial average used upon detection is sup- 
pressed and a statistical similarity can be calculated with a 
higher degree of accuracy. Accordingly, the accuracy in 
detection of electronic watermark data is improved. 

It is to be noted that the identification data insertion 
system and the identification data detection system for 
digital data may be combined so as to form a single 
identification data insertion and detection system for digital 
data. 

The above and other objects, features and advantages of 
the present invention will become apparent from the fol- 
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lowing description and the appended claims, taken in con- 
junction with the accompanying drawings in which like 
parts or elements are denoted by like reference characters. 

5 BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a block diagram of an electronic watermark data 
insertion system to which the present invention is applied; 

FIG. 2 is a block diagram of an electronic watermark data 
1Q detection system to which the present invention is applied; 

FIG. 3 is a block diagram of another electronic watermark 
data insertion system to which the present invention is 
applied; 

FIG. 4 is a block diagram of another electronic watermark 
is data detection system to which the present invention is 
applied; 

1 FIG. 5 is a block diagram showing a conventional elec- 
tronic watermark data insertion system; 
FIG. 6 is a block diagram showing a conventional elec- 
20 tronic watermark data detection system; 

FIG. 7 is a block diagram showing another conventional 
electronic watermark data insertion system; and 
FIG. 8 is a block diagram showing another conventional 
25 electronic watermark data detection system. 

DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

Referring first to FIG. 1, there is shown an electronic 

30 watermark data insertion system to which the present inven- 
tion is applied. The electronic watermark data insertion 
system shown includes a DCT calculator 103 for extracting 
blocks 102 of 8x8 pixels from within a current image 101 
which makes an object of insertion of electronic watermark 

35 data and performing DCT (discrete cosine transform) cal- 
culation for the blocks 102, a quantizer 104 for quantizing 
data outputted from the DCT calculator 103, a quantization 
table 105 for being referred to when the quantizer 104 
performs quantization, an electronic watermark data inser- 

40 tion element 107 for inserting electronic watermark data 106 
into data after quantization, and an encoder 113 for encoding 
data outputted from the electronic watermark data insertion 
element 107 to produce MPEG data 114 in which the 
electronic watermark data 106 are inserted and outputting 

45 the MPEG data 114. 

The electronic watermark data insertion element 107 
includes a partial average calculator 108 for calculating a 
partial average of three neighboring points of data outputted 
from the quantizer 104 in accordance with avg(f(i))«(|f(i- 

50 l)|+|f(i)|+|f(i+l)|)/3, a multiplier 110 for multiplying outputs 
of the electronic watermark data 106 and the partial average 
calculator 108 for individual factors, another multiplier 111 
for multiplying each factor of an output of the multiplier 110 
by a constant 109 which is used to vary the size of the 

55 electronic watermark data to be inserted, and an adder 112 
for adding an output of the multiplier 111 and the output of 
the quantizer 104 for individual factors. 

The electronic watermark data insertion element 107 

60 performs calculation defined by the following expression: 

where f(i) is the data outputted from the quantizer 104, 
avg(f(i)) is the data outputted from the partial average 
65 calculator 108, w(i) is the electronic watermark data 106, the 
constant is the constant 109, and i is the number of each 
factor after zigzag scanning of a block of 8x8 pixels. 
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Referring now to FIG. 2, there is shown an electronic 
watermark data detection system to which the present inven- 
tion is applied. The electronic watermark data detection 
system detects electronic watermark data inserted in an 
image by the electronic watermark data insertion system of 
FIG. 1 described above. The electronic watermark data 
detection system includes a decoder 202 for decoding 
MPEG data 201 which make ao object of detection of 
electronic watermark data, a dequantizer 203 for dcquantiz- 
ing data outputted from the decoder 202, an inverse DCT 
calculator 204 for performing inverse DCT calculation for 
data outputted from the dequantizer 203 to obtain image data 
205 and outputting the image data 205, an electronic water- 
mark data extractor 206 for extracting, from among data 
outputted from the decoder 202, those data which are 
supposed to be electronic watermark data, an adder 209 for 
cumulatively adding data in units of a matrix block of 8x8 
pixels outputted from the electronic watermark data extrac- 
tor 206 for individual factors over one screen, and an inner 
product calculator 210 for calculating inner products of 
electronic watermark data 211 to be detected and data 
outputted from the adder 209 to obtain a statistical similarity 
212 and outputting the statistical similarity 212. 

The electronic watermark data extractor 206 includes a 
partial average calculator 207 for calculating a partial aver- 
age of three neighboring points of data in units of a block of 
8x8 pixels outputted from the decoder 202 in accordance 
with avg(F(i))=(|F(i-l)|+|F(i)|+|F(i+l)|)/3, and a divider 208 
for dividing data F(i) outputted from the decoder 202 by a 
partial average avg(F(0) outputted front the partial average 
calculator 207. 

The electronic watermark data extractor 206 performs 
calculation defined by the following expression: 

where F(i) is the data outputted from the decoder 202, 
avg(F(i)) is the partial average outputted from the partial 
average calculator 207, W(i) is the data which are supposed 
to include electronic watermark data outputted from the 
electronic watermark data extractor 206, and i is the number 
of each factor of a block of 8x8 pixels. 

In the following, operation for insertion and detection of 
electronic watermark data of the electronic watermark data 
insertion and detection systems described above with refer- 
ence to FIGS. 1 and 2 is described in detail with reference 
to FIGS. 1 and 2. 

First, operation for insertion of electronic watermark data 
is described. 

A current image 101 is extracted for each block of 8x8 
pixels by ordinary MPEG compression processing, and the 
DCT calculator 103 performs DCT calculation for the thus 
extracted data. Data of frequency components obtained by 
the DCT calculation of the DCT calculator 103 are quantized 
with reference to the quantization table 105 by the quantizer 
104. The electronic watermark data insertion element 107 
embeds electronic watermark data 106 into the data output- 
ted from the quantizer 104. In this instance, the electronic 
watermark data insertion element 107 performs processing 
equivalent to calculation of the following expression: 

F(i>/(i>avg(/(0)xMi>constant 

where i is the number of a factor of a frequency component 
of the block of 8x8 pixels, w(i) is the value of each factor 
of the electronic watermark data 106, the constant is the 
constant 109, f(i) is each of the factors of the 8x8 pixel 
block, avg(f(i)) is the average of absolute values of three 
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neighboring factors, that is, (|f(i-l)|+|f(i)|+|f(i+l)D/3> and 
F(i) is the new frequency component in which the electronic 
watermark data 106 are inserted. 

The encoder 113 encodes the data F(i) outputted from the 
5 electronic watermark data insertion element 107 by MPEG 
processing to produce MPEG data 114 in which the elec- 
tronic watermark data 106 are embedded. 

Now, operation for detection of electronic watermark data 
is described. 

10 First, MPEG data 201 which make an object of detection 
is subject to MPEG decoding by the decoder 202. The data 
decoded by the decoder 202 are dequantized by ordinary 
MPEG decoding processing by the dequantizer 203 and 
inverse DCT calculated by the inverse DCT calculator 204 

15 to produce reproduction image data 205. 

The electronic watermark data extractor 206 extracts the 
data decoded by the decoder 202, performs calculation of 
F(i)/avg(F(i)) and extracts those data which are supposed to 
be embedded electronic watermark data. Here, F(i) is the 

20 frequency component of a decoded block of 8x8 pixels, and 
avg(F(i)) is the partial average ((|F(i-l)|+|F(i)|+|F(i+l)|/3. 
The adder 209 calculates a sum total of the extracted data in 
units of 8x8 pixels outputted from the electronic watermark 
data extractor 208 for individual factors over one screen. 

25 After the sum totals of the extracted data of the one screen 
are calculated, the inner product calculator 210 calculates 
inner products of the sum totals and electronic watermark 
data 211 to be detected to produce a statistical similarity 212 
and outputs the statistical similarity 212. 

30 If the statistical similarity 212 is higher than a particular 
fixed value, then it is discriminated that electronic water- 
mark data equivalent to the electronic watermark data 211 
are embedded in the MPEG data 201, 
In this instance, if the electronic watermark data 211 

35 represent inhibition of duplication, then the reproduction 
apparatus in which the electronic watermark data detection 
system is incorporated can take such a countermeasure as to 
inhibit duplication of the image data 205 produced therein, 
Referring now to FIG. 3, there is shown another electronic 

40 watermark data insertion system to which the present inven- 
tion is applied. The electronic watermark data insertion 
system includes a DCT calculator 303 for extracting matrix 
blocks 302 of 8x8 pixels from within a current image 301 
which makes an object of insertion of electronic watermark 

45 data and performing DCT calculation for the blocks 302, an 
electronic watermark data insertion element 305 for insert- 
ing electronic watermark data 304 into data for which the 
DCT transform has been performed, a quantizer 311 for 
quantizing data outputted from the electronic watermark 

50 data insertion element 305, a quantization table 312 for 
being referred to when the quantizer 104 performs quanti- 
zation and when the electronic watermark data insertion 
element 305 calculates a partial average, and an encoder 313 
for encoding data outputted from the quantizer 311 and 

55 outputting MPEG data 314 in which the electronic water- 
mark data 304 are embedded. 

The electronic watermark data insertion element 305 
includes a partial average calculator 307 for calculating a 
partial average of data outputted from the DCT calculator 

60 303 with reference to the quantization table 312, a multiplier 
308 for multiplying the output of the partial average calcu- 
lator 307 by the electronic watermark data 304 for individual 
factors, another multiplier 309 for multiplying the data 
outputted from the multiplier 308 by a constant a 306 which 

65 is used to vary the size of the electronic watermark data to 
be inserted, and an adder 310 for adding the data outputted 
from the DCT calculator 303 and the data outputted from the 
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multiplier 309 for individual factors. The electronic water- 
mark data insertion element 305 thus performs processing 
equivalent to calculation of the following expression: 

where f(i) is the data outputted from the DCT calculator 303, 
avg(f(i)) is the data outputted from the partial average 
calculator 307, w(i) is the electronic watermark data 304, the 
constant is the constant a 306, and i is the number of a factor 



In the following, operation for insertion and detection of 
electronic watermark data of the electronic watermark data 
insertion and detection systems described above with refer- 
ence to FIGS. 3 and 4 is described in detail with reference 
to FIGS. 3 and 4. 

First, operation for insertion of electronic watermark data 
is described. 

A current image 301 is extracted for individual blocks 302 
of 8x8 pixels by ordinary MPEG compression processing, 
after the data outputted from the DCT calculator 303 are i° and tnc DCT calculator 303 performs DCT calculation for 



the thus extracted data. The electronic watermark data 
insertion element 305 embeds electronic watermark data 304 
into the data of frequency components obtained by the DCT 
transform. In this instance, the electronic watermark data 
15 insertion element 305 performs processing equivalent to 
calculation of the following expression: 

^(0 *A0 +ay g(/(0) x ^CO 5 * 00 m 13111 



where i is the factor number after zigzag scanning of a block 
of 8x8 pixels, f(i) is the value of each coefficient obtained by 
the DCT transform, w(i) is the value of each factor of the 
electronic watermark data 304, and avg(f(i)) is the average 
of absolute values of three neighboring points of each factor. 
Here, avg(f(i)) is calculated in accordance with the follow- 
ing expressions: 

V((-l)=|integral part of (fti-l)/(?(i-l))x{?(/-l)| 

VCO-|intcgral part of .(KWO^CCOl 

V(i+l)-|iniegral part of (f{i+i)/Q(i+l))xQ(i+l)\ 

avg(/(i))-(V(i-l)+V(i>VT[i+l))/3 

The quantizer 311 refers to the quantization table 312 to 



zigzag scanned. 

Here, the partial average calculator 307 in the electronic 
watermark data insertion clement 305 performs, in order to 
calculate a partial average avg(f(i)) of three neighboring 
points of the data outputted from the DCT calculator 303, 
processing equivalent to calculation of the following expres- 
sions: 

V((-l)-(inlcgral part of (/(r-l)/fi(/-l)>0(i-l)| 

V(0-|integral part of (fT0«))*G(0l 20 
V(i+l)-[intcgral part of (ft'+l)/{20'+l))xe(i+l)| 
avg(/(0)-(n'"-l)+nO+V(''+l))/3 

Referring now to FIG. 4, there is shown another electronic 25 
watermark data detection system to which the present inven- 
tion is applied. The electronic watermark data detection 
system detects electronic watermark data inserted by the 
electronic watermark data iasertion system described above 
with reference to FIG. 3. To this end, it includes a decoder 30 
402 for decoding MPEG data 401 which make an object of 
detection of electronic watermark data, a dequantizer 403 for 
dequantizing data outputted from the decoder 402, an 
inverse DCT calculator 404 for performing inverse DCT 

calculation for data outputted from the dequantizer 403 to 35 perform quantization processing for the data outputted from 
obtain image data 405 and outputting the image data 405, an the electronic watermark data insertion element 305, and the 
electronic watermark data extractor 406 for extracting, from encoder 313 performs encoding of the data outputted from 
among the data outputted from the dequantizer 403, those the quantizer 311 to produce MPEG data 314. 
data which are supposed to be electronic watermark data and Now, operation for detection of electronic watermark data 

outputting the extracted data, an adder 409 for adding data 40 is described. 

in matrix units of an 8x8 pixel block outputted from the First, MPEG data 401 which make an object of detection 
electronic watermark data extractor 406 for individual fac- are subject to MPEG decoding processing by the decoder 
tors over one screen, and an inner product calculator 410 for 402. The data decoded by the decoder 402 are dequantized 
calculating inner products of data outputted from the adder by ordinary MPEG decoding processing by the dequantizer 
409 and electronic watermark data 411 to be detected to 45 403, and inverse DCT calculated by the inverse DCT cal- 
obtain a statistical similarity 412 and outputting the statis- culator 404 to produce image data 405. 
tical similarity 412. The electronic watermark data extractor 406 extracts 

The electronic watermark data extractor 406 includes a decoded data from the dequantizer 403, performs calculation 
partial average calculator 407 for calculating a partial aver- of F(i)/avg(F(i)) and extracts data supposed to be embedded 
age of three neighboring points of data in units of 8x8 pixel 50 electronic watermark data. Here, F(i) is the value of each 
block outputted from the dequantizer 403 in accordance with factor after zigzag scanning in units of an 8x8 pixel block 
avg(F(i))=(|F(i-l)|+|F(i)|+|F(i+l)|)/3, and a divider 408 for outputted from the dequantizer 403, and avg(F(i)) is the 
dividing the data F(i) outputted from the dequantizer 403 by partial average (|F(i-l)|+|F(i)|+|F<(i+l)|)/3. The adder 409 
the partial average avg(F(i)) outputted from the partial calculates sum totals of the extracted data in units of an 8x8 
average calculator 407. Here, F(i) is the frequency compo- 55 pixel block outputted from the electronic watermark data 

extractor 406 for individual factors over one screen. After 
the sum totals of the extracted data for one screen are 
calculated, the inner product calculator 410 calculates inner 
products of the sum totals and the electronic watermark data 
60 411 to be detected to obtain a statistical similarity 412 and 
outputs the statistical similarity 412. 

If the statistical similarity 412 is higher than a particular 
fixed value, then it is discriminated that electronic water- 
mark data equivalent to the electronic watermark data 411 
from the partial average calculator 407, and W(i) is the data 65 are embedded in the MPEG data 401. 
outputted from the electronic watermark data extractor 406 In this instance, if the electronic watermark data 411 
and supposed to include electronic watermark data. represent inhibition of duplication, then the reproduction 



nent of an 8x8 pixel block outputted from the dequantizer 
403, and i is the number of each factor after zigzag scanning 
of the 8x8 pixel block. 

The electronic watermark data extractor 406 performs 
calculation in accordance with the following expression: 

vno^co/avgc^o) 

where F(i) is the data outputted from the inverse DCT 
calculator 404, avg(F(i)) is the partial average outputted 
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apparatus in which the electronic watermark data detection 
system is incorporated can take such a countermeasure as to 
inhibit duplication of the image data 405 produced therein. 

Having now fully described the invention, it will be 
apparent to one of ordinary skill in the art that many changes s 
and modifications can be made thereto without departing 
from the spirit and scope of the invention as set forth herein. 

What is claimed is: 

1. An electronic watermark insertion apparatus, compris- 
ing: 10 

conversion means for performing discrete cosine trans- 
form for an original image to convert the original image 
into data f(i) of frequency components, i being 1, 
2, . . . , n; 

electronic watermark insertion means for multiplying a 15 
partial average of the data f(i) and electronic watermark 
data w(i) determined in advance for the individual ' 
factors (i), i being 1, 2, . . . , n, and adding the data f(i) 
to the results obtained by multiplying the results of the 
multiplication by a constant to determine an electronic 20 
watermark containing data F(i), i being 1, 2, . . . , n; 

quantization means for quantizing the data F(i) with a 
quantization step sizes Q(i), i being 1, 2, , . . , n; and 

coding means for compression coding the data after the 2 s 
quantization; 

the partial average of the data f(i) being given by (V(i- 
l)+V(i)+V(i+l))/3 where V(i-l)=|integral part of (f(i- 
l)/Q(i-l)xQ(i-l)|, V(i)-|imcgral part of (f(i)/Q(i)xQ 
(i)|, and V(i+l)»|integral part of (f(i+l)/Q(i+l)x 30 
Q(i + l)|. 

2. An electronic watermark detection apparatus to which 
data obtained by performing a discrete cosine transform for 
an original image to convert the original image into data f(i) 

of frequency components, i being 1, 2, . . . , n, multiplying 35 
a partial average of the data f(i) and electronic watermark 
data w(i) determined in advance for the individual factors 
(i), i being 1, 2, . . . , n, and adding the data f(i) to the results 
obtained by multiplying the results of the multiplication by 
a constant to determine electronic watermark containing 40 
data F(i), i being 1, 2, . . . , n, quantizing the data F(i) with 
quantizaiton step sizes Q(i), i being 1, 2, . . . , n, and then 
compression coding the data after the quantization, the 
partial average of the data f(i) being given by 

45 

(V(i-l>+V(0+V(»+l))/3 

where V(i-l>| integral part of (f(i-l)/Q(i-l)xQ(i-l)|, V(i)- 
|integral part of (f(i)/Q(i)xQ(i)|, and V(i+l)=|integral part of 
(fi+l)/Q(i+l)xQ(i+l)|, the apparatus comprising: 5Q 
decoding means for decoding the compression coded 
data; 

dequantization means for dequantizing data F(i) obtained 

by the decoding, i being 1, 2, . . . , n; 
inverse discrete cosine transform means for performing an 55 

inverse discrete cosine transform for data obtained by 

the dequantization in- order to restore the original 

image; and 

electronic watermark detection means for calculating a 
statistical similarity between values obtained by divid- 60 
ing the data F(i) by a partial average (|F(i-l)|+|F(i)|+ 



|F(i+l)|)/3 of the data F(i) and the electronic watermark 
data to discriminate whether or not the electronic 
watermark is included in the data supplied to said 
electronic watermark detection apparatus. 

3. An electronic watermark insertion method, comprising: 
a conversion step of performing a discrete cosine trans- 
form for an original image in order to convert the 
original image into data f(i) of frequency components, 
i being 1, 2, . . . , n; 

an electronic watermark insertion step of multiplying a 
partial average of the data f(i) and electronic watermark 
data w(i) determined in advance for the individual 
factors (i), i being 1, 2, . . . , n, and adding the data f(i) 
to the results obtained by multiplying the results of the 
multiplication by a constant to determine an electronic 
watermark containing data F(i), i being 1, 2, . . . , n; 

a quantization step of quantizing the data F(i) with 
quantization step sizes Q(i), i being 1, 2, . . . , n; and 

a coding step of compression coding the data after the 
quantization; 

the partial average of the data f(i) being given by (V(i- 
l)+V(i)+V(i+l))/3 where V(i-l)-|inlegral part of (f(i- 
l)/Q(i-l)xQ(i-l)|, and V(i+l)-|integral part of (f(i+l)/ 
Q(i+l)xQ(i-l)|. 

4. An electronic watermark detection method to which 
data obtained by performing a discrete cosine transform for 
an original image to convert the original image into data f(i) 
of frequency components, i being 1, 2, . . . , n, multiplying 
a partial average of the data f(i) and electronic watermark 
data w(i) determined in advance for the individual factors 
(i), i being 1, 2, . . . , n, and adding the data f(i) to the results 
obtained by multiplying the results of the multiplication by 
a constant in order to determine an electronic watermark 
containing data F(i), i being 1, 2, . . . , n, quantizing the data 
F(i) with quantization step sizes Q(i), i being 1, 2, . . . , n, 
and then compression coding the data after the quantization, 
the partial average of the data f(i) being given by 

(V(i-l)+K( t >V(i+l))/3 

where V(i-l)-|integral part of (f(i-l)/Q(i)xQ(i-l)|, V(i)- 
integral part of (f(i)/Q(i)xQ(i)|, and V(i+l)-|integral part of 
(f(i+l)/Q(i+l)xQ(i+l)|, the method comprising: 

a decoding step of decoding the compression coded data; 
a dequantization step of dequantizing the data (i) obtained 

by the decoding, i being 1, 2, . . . , n; 
an inverse discrete cosine transform step of performing an 

inverse discrete cosine transform for data obtained by 

the dequantization in order to restore the original 

image; and 

an electronic watermark detection step of calculating a 
statistical similarity between values obtained by divid- 
ing the data F(i) by a partial average (|F(i-l)|+|F(i)|+ 
|F(i+l)|)/3 of the data F(i) and the electronic watermark 
data to discriminate whether or not the electronic 
watermark is included in the data supplied to the 
electronic watermark detection method. 
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